Method and device applying artificial intelligence to send money by using voice input

ABSTRACT

An example device includes a memory configured to store at least one program; a microphone configured to receive a voice; and at least one processor configured to execute the at least one program to control the device to perform operations for sending money to a recipient. The operations include determining a payment intention of a user based on analyzing the received voice input; retrieving contact information from a stored contact list based on the name of the recipient; transmitting the name and the contact information of the recipient to a bank server together with an amount of money specified in the voice input; receiving remittance details from the bank server; and approving the remittance details. The device may analyze the received voice input by using an artificial intelligence (AI) algorithm.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 U.S.C. Section119 to Korean Patent Application No. 10-2016-0154879, filed on Nov. 21,2016, and to Korean Patent Application No. 10-2017-0132758, filed onOct. 12, 2017, in the Korean Intellectual Property Office, thedisclosures of each of which are incorporated by reference herein intheir entireties.

BACKGROUND 1. Field

The present disclosure generally relates to a method and device forsending money using voice input.

The present disclosure also relates to an artificial intelligence (AI)system and its application that simulates functions, such as recognitionand determinations of the human brain, using a machine learningalgorithm.

2. Description of Related Art

As multimedia technology and network technology have developed, a usermay receive various services using a device. In particular, as speechrecognition technology has developed, a user may input his or her voiceto the device, and the device may perform an operation according to theuser's voice (e.g., according to commands spoken by the user).

A user may access a financial service using a device executing anapplication provided by a bank. For example, the user may send money toan account of a recipient by using the device. The user may execute theapplication, input an account number, a password, etc., and send moneyto the account of the recipient.

Also, in recent years, an artificial intelligence (AI) system thatimplements human-level intelligence has been used in various fields. AnAI system is a machine-learning system that learns for itself, makesdeterminations, and becomes “smarter”, unlike existing rule-basedsystems. The AI system may provide an improved recognition rate andunderstand user preferences more accurately as it is used, and thusexisting rule-based systems are increasingly being replaced bydeep-learning based AI systems.

AI technology includes machine learning (e.g., deep learning) andelement technologies that utilize machine learning.

Machine learning is an algorithm technology in which the machine itselfclassifies/learns characteristics of input data. Element technology istechnology that simulates functions such as recognition anddeterminations of the human brain using machine-learning algorithms suchas deep learning and includes technical fields such as linguisticunderstanding, visual understanding, inference/prediction, knowledgerepresentation, motion control, etc.

AI technology has been applied to various fields. Linguisticunderstanding is technology for recognizing and applying/processinghuman language/characters and includes natural language processing,machine translation, dialogue system, query/response, speechrecognition/synthesis, and the like. Visual understanding is technologyfor recognizing and processing objects as human vision, and includesobject recognition, object tracking, image search, human recognition,scene understanding, spatial understanding, image enhancement, and thelike. Inference/prediction is technology for determining and logicallyinferring and predicting information, and includesknowledge/probability-based inference, optimization prediction,preference-based planning, recommendation, and the like. Knowledgerepresentation is technology for automating experience information ofhumans into knowledge data and includes knowledge building(generation/classification of data), knowledge management (utilizationof data), and the like. Motion control is technology for controlling anautonomous travel of a vehicle and motion of a robot, and includesmotion control (navigation, collision, and traveling), operation control(behavior control), and the like.

SUMMARY

A method and a device for sending money to an account of a recipient byusing voice are provided.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the disclosed embodiments.

According to an aspect of an example embodiment, a device includes amemory configured to store at least one program; a microphone configuredto receive a voice input; and at least one processor configured toexecute the at least one program to perform operations for sending moneyto a recipient, wherein the operations include determining a paymentintention of a user based on analyzing the received voice input;retrieving contact information from a stored contact list based on aname of the recipient; transmitting the name and the contact informationof the recipient to a bank server together with an amount of moneyspecified in the voice input; receiving remittance details from the bankserver; and approving the remittance details

According to an aspect of another example embodiment, a paying methodincludes receiving a voice input of a user; determining a paymentintention of a user based on an analysis of the received voice input;retrieving contact information from a stored contact list based on aname of a recipient specified in the voice input; transmitting the nameand the contact information of the recipient to a bank server togetherwith an amount specified in the voice input; receiving remittancedetails from the bank server; and approving the remittance details.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects, features and attendant advantages of thepresent disclosure will become apparent and more readily appreciatedfrom the following detailed description, taken in conjunction with theaccompanying drawings, in which like reference numeral refer to likeelements and wherein:

FIG. 1 is a diagram illustrating a method by which a user sends money byusing a user's voice, according to an example embodiment;

FIG. 2 is a block diagram illustrating a device according to an exampleembodiment;

FIG. 3 is a diagram illustrating a device learning a pattern accordingto an example embodiment;

FIG. 4 is a diagram illustrating a method of approving remittancedetails, according to an example embodiment;

FIG. 5 is a diagram illustrating a method of selecting one of aplurality of recipients, according to an example embodiment;

FIG. 6 is a diagram illustrating a method of selecting any of aplurality of banks, according to an example embodiment;

FIG. 7 is a flowchart illustrating a method of sending money by usingvoice, according to an example embodiment;

FIG. 8 is a diagram illustrating a method of paying by using voiceaccording to another example embodiment;

FIG. 9 is a diagram illustrating a device learning a payment patternaccording to an example embodiment;

FIG. 10 is a flowchart illustrating a method of paying using voice,according to an example embodiment;

FIG. 11 is a block diagram of a processor according to some exampleembodiments;

FIG. 12 is a block diagram of a data learner according to some exampleembodiments;

FIG. 13 is a block diagram of a data recognizer according to someexample embodiments;

FIG. 14 is a diagram illustrating an example of learning and recognizingdata by interaction between a device and a server according to someexample embodiments; and

FIGS. 15 and 16 are flowcharts of a network system using a datarecognition model according to some example embodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to various non-limitingembodiments, examples of which are illustrated in the accompanyingdrawings. In the drawings, parts irrelevant to the description areomitted to clearly describe the example embodiments, and like referencenumerals denote like elements throughout the specification. In thisregard, the example embodiments may have different forms and should notbe construed as being limited to the descriptions set forth herein.Accordingly, the example embodiments are described below, by referringto the figures, to explain aspects of the present disclosure. As usedherein, the term “and/or” includes any and all combinations of one ormore of the associated listed items. Expressions such as “at least oneof,” when preceding a list of elements, modify the entire list ofelements and do not modify the individual elements of the list.

Throughout the present disclosure, when it is described that a certainpart is “connected” to another part, it should be understood that thecertain part may be “directly connected” to another part or“electrically connected” to another part via another element in themiddle. Also, when a component “includes” an element, unless there isanother opposite description thereto, it should be understood that thecomponent does not exclude another element, but may further includeanother element.

Hereinafter, the present disclosure will be described in detail withreference to the accompanying drawings.

FIG. 1 is a diagram illustrating a method by which a user sends money byusing the user's voice, according to an example embodiment. Referring toFIG. 1 , the user may input his or her voice to a device 10 by speaking(e.g., into a microphone) in order to send the money to a recipient. Inparticular, the user may send the money to a recipient by speaking onlyby a name of the recipient without speaking or inputting an accountnumber of the recipient.

The device 10 may receive voice input from the user. The device 10 mayinclude a microphone, which receives the user's voice. The device 10 mayreceive the voice input of the user via the microphone by executing, forexample, a voice assistant application such as “S Voice” and controllingthe executed application.

The device 10 may recognize the user's voice as indicated at item 1 inFIG. 1 . The device 10 may analyze the voice to determine an intentionof the user. For example, if the device receives a voice input of theuser saying ‘send 100 million won to Samsung’, the device 10 maydetermine from the user's voice whether the user intends to send money.In an example embodiment, the device 10 may store in memory the entireuser voice input when the user sends money and use the storedinformation to learn a pattern of the voice input when sending money.The device 10 may determine the intention of the user more accuratelythrough learning. At the beginning of learning, when the user's voice isinput, the device 10 may confirm whether to send money. The device 10may more accurately determine the sending intention of the user throughrepeated learning.

As an example, the device 10 may compare a stored voice pattern with thepattern of the input voice to determine the intention of the user. Thestored voice pattern can include the pattern of the user's voice inputwhen the user intends to send money. The device 10 may determine thatthe user intends to send money if the stored voice pattern is similar oridentical to the pattern of the input voice (e.g., the similarity isequal to or exceeds a threshold similarity). The pattern of the storedvoice may be updated or added to through learning.

The device 10 may confirm the name of the recipient or a title andsearch for the name or the title stored in a contact list. For example,if the user inputs the recipient as ‘Samsung’, the device 10 may searchfor ‘Samsung’ in the contact list. For example, the device 10 mayconfirm a phone number of ‘Samsung’ in the contact list.

The device 10 may transmit user information, recipient information, andan amount of money to a bank server 20 as indicated at item 2 in FIG. 1. The user information includes, without limitation, a name of the user,an account number, and the like. The recipient information includes,without limitation, the name of the recipient, a telephone number, andthe like. The recipient information may not include an account number ofthe recipient. The amount of money indicates an amount of moneyspecified in the user's voice input, and is an amount of money that theuser will send to the recipient.

The device 10 may be, without limitation, a smart phone, a tablet PC, aPC, a smart TV, a mobile phone, a personal digital assistant (PDA), alaptop, a media player, a micro server, a global positioning system(GPS) device, an e-book terminal, a digital broadcasting terminal, anavigation system, a kiosk, an MP3 player, a digital camera, consumerelectronics, and other mobile or non-mobile computing devices. Thedevice 10 may also be a wearable device, such as, without limitation, awatch, glasses, a hair band, a ring, and the like having a communicationfunction and a data processing function. The device 10 may include anykind of device capable of receiving voice input of a user and providinga reply message to the user.

Also, the device 10 may communicate with other devices (not shown) overa network in order to use various types of context information. Thenetwork may include a local area network (LAN), a wide area network(WAN), a value added network (VAN), a mobile radio communicationnetwork, a satellite communication network, and/or a combinationthereof, may be a data communication network in a comprehensive sensefor allowing respective network elements to smoothly communicate witheach other, and may include wired Internet, wireless Internet, and amobile wireless communication network. Wireless communication mayinclude, for example, Wi-Fi, Bluetooth, Bluetooth low energy, ZigBee,Wi-Fi Direct (WFD), ultra wideband (UWB), infrared data association(IrDA), Near Field Communication (NFC), and the like, but is not limitedthereto.

The bank server 20 may receive the user information and the recipientinformation as indicated at item 3 in FIG. 1 . The bank server 20 maysearch for an account number that matches the user information. The bankserver 20 may search for the account number of the user by using, forexample, the name of the user and the telephone number. Also, the bankserver 20 may search for an account number assigned (or matched) tounique identification information of the device 10. The device 10 mayinclude unique identification information and the bank server 20 may usethe unique identification information of the device 10 to search anaccount database for the account number of the user of the device 10.The bank server 20 may also search for an account number matching therecipient information. For example, the bank server 20 may search forthe account number matching the name and the telephone number of therecipient.

The bank server 20 may generate remittance details as indicated at item4 in FIG. 1 . The bank server 20 may generate the remittance detailsincluding, without limitation, the account number of the user, the nameof the recipient, the account number of the recipient, and the amount ofmoney. For example, the bank server 20 may generate remittance details‘send 10 thousand won from bank A, 11-1111 (account number), and AAA(user name) to bank B, 22-2222 (account number), and BBB (recipientname)’.

The bank server 20 may transmit the remittance details to the device 10.

The device 10 may display the remittance details. The device 10 maydisplay the remittance details to allow the user to confirm whether theintention of the user's voice input and the remittance details coincidewith each other.

The user may approve the remittance details. The user may input, forexample, one or more of voice, a fingerprint, an iris scan, a veinimage, a face image, and a password if the user wants to send moneyaccording to remittance details. The device 10 may performauthentication by determining whether the input voice, fingerprint, irisscan, vein image, face image and/or password matches (or match) personalinformation of the user. This authentication of remittance details isshown as item 5 in FIG. 1 .

The device 10 may transmit an authentication result to the bank server20 as shown in item 6 in FIG. 1 .

The bank server 20 may receive the authentication result and send themoney to the recipient according to a received authentication resultauthenticating the remittance details as shown at item 7 in FIG. 1 . Thebank server 20 may send the money to the recipient if the user isauthenticated as a legitimate user (and optionally send confirmation tothe device 10 that the money is sent), and may not send the money andtransmit an error message to the device 10 otherwise.

FIG. 2 is a block diagram illustrating the device 10 according to anexample embodiment. Referring to FIG. 2 , the device 10 may include aprocessor 11, a memory 12, a display 13, and a microphone 14.

The processor (e.g., including processing circuitry such as a CPU and/ordedicated hardware circuitry) 11 may control the overall operation ofthe device 10 including the memory 12, the display 13, and themicrophone 14. The processor 11 may control storing data in and/orreading data to/from the memory 12. The processor 11 may determine animage to be displayed on the display 13 and may control the display 13to display the image. The processor 11 may control turning themicrophone 14 on/off and analyze (e.g., by executing a voice analysisapplication) a voice input through the microphone 14.

The memory (e.g., ROM, RAM, memory card, nonvolatile, volatile, solidstate, hard disk, and the like) 12 may store personal information of auser, biological information, and the like. For example, the memory 12may store, without limitation, a user's voice, fingerprint, iris scan,vein image, facial image, and/or a password. The memory 12 may storesamples of the user's voice and/or prior voice input for analyzing apattern of the user's voice.

The display (e.g., LCD, OLED, and the like) 13 may display images andreproduce video content under control of the processor 11.

The microphone 14 may receive voice input. The microphone 14 may includecircuitry to convert a sound (e.g., voice input) generated in aperiphery of the device 10 into an electric signal and output theelectric signal to the processor 11.

FIG. 3 is a diagram illustrating the device 10 learning a patternaccording to an example embodiment. Referring to FIG. 3 , the device 10may, for example, execute a voice analysis application for analyzingvarious types of sentences and learn patterns based thereon.

A user may say various types of sentences to send money. For example, inorder to send 100 million won from a bank account of the user to Samsung(a recipient), the user may say the following type of sentences:

1. Send 100 million won from the “A” bank account to Samsung

2. Send Samsung 100 million won

3. Send 100 million to Samsung

The device 10 may analyze and learn a pattern of the user's voice toidentify sentences including the user's intention to send money.

When the user has a plurality of accounts, the device 10 may confirmwith the user the account among the plurality of accounts from which towithdraw money. Once an account is designated, the money transfersinitiated using device 10 may withdraw money from the designated accountfrom then on, unless there is a different instruction from the user.

FIG. 4 is a diagram for illustrating a method of approving remittancedetails, according to an example embodiment. A user may approve theremittance details by using, without limitation, voice input,fingerprint, vein image, face image or iris scan.

The device 10 may receive the remittance details from the bank server 20and display the remittance details on the display 13 thereof. Theremittance details may include, without limitation, an account number ofthe user, an account number of a recipient, and an amount of money.

The user may approve the remittance details after visually confirmingthe displayed remittance details. When the user approves the remittancedetails, the user may use a voice input, fingerprint, vein image, facialimage or iris scan. The device 10 may transmit a message indicating thatthe remittance details are approved to the bank server 20 if an inputvoice, fingerprint, or iris matches (e.g., has similarity equal toexceeding a predetermined similarity threshold) the user's voice input,fingerprint, vein image, facial image or iris scan as reflected ininformation stored in memory 12 of the device.

FIG. 5 is a diagram illustrating a method of selecting one of aplurality of recipients according to an example embodiment. A user mayselect any one of the plurality of recipients through, for example,voice input.

The device 10 may search a contact list stored in memory 12 (or someother external memory) for a name identified as a recipient. If aplurality of recipients including the identified name are found in thecontact list, the device 10 may display names of the plurality of foundrecipients on display 13. The user may select any one of the displayednames by voice input.

A case in which the following two recipients are found by a name ofSamsung is used as an example.

-   1. Samsung Electronics-   2. Samsung Corporation

The device 10 may display the two recipients on the display 13. The usermay select either a 1^(st) recipient or a 2^(nd) recipient by voiceinput. For example, the user may select a recipient by inputting a voicesuch as ‘send money to the 1^(st) one’ or ‘send money to SamsungElectronics’. If display 13 is configured as a touch screen, the usermay use a touch input to select a recipient.

FIG. 6 is a diagram for illustrating a method of selecting any of aplurality of banks according to an example embodiment. A user may selectany bank (or an account number) from the plurality of banks (or accountnumbers) using voice input.

The bank server 20 may transmit the plurality of banks (or accountnumbers) registered in a name of a recipient when transmittingremittance details to the device 10. For example, if there are aplurality of account numbers registered in the name of the recipient,the device 10 may display the account numbers to the user on display 13for the user to determine to which account number to send money. Asabove, the user may select any one of the displayed account numbers byvoice or touch input.

The following two account numbers are found under a name of Samsung, forexample.

-   1. Bank A (33-3333)-   2. Bank B (55-5555)

The device 10 may display the two account numbers on the display 13. Theuser may select either a 1^(st) or 2^(nd) account number by voice input.For example, the user may select a bank or an account number by a voiceinput providing by speaking, for example, ‘Send money to Bank A’, ‘Sendmoney to the 1^(st) one’, or ‘Send money to the 55-5555 account’.

FIG. 7 is a flowchart illustrating a method of sending money by usingvoice input, according to an example embodiment. Referring to FIG. 7 , auser may input a name of a recipient and an amount of money by voiceinput and send money to the recipient.

In operation 710, the device 10 may receive the user's voice input tomicrophone 14.

In operation 720, the device 10 may analyze the received voice input todetermine an intention of the user to send money. As a result ofanalyzing the received voice, if it is determined that there is nointention to send money, the device 10 does not perform a process forsending money. The voice input may include the name of the recipient andthe amount of money, etc. For example, the device 10 may analyze thevoice input and determine that the user intends to send money if aninstruction, the name, the amount of money, and the like are included inthe voice input.

In operation 730, the device 10 may search in a stored contact list fora contact corresponding to the name of the recipient. If a contactcorresponding to the name of the recipient is not found, the device 10may display on the display 13 an information indicating that the contactis missing/not found. The user may input contact information for therecipient by voice input. The device 10 may store the name of therecipient and the corresponding contact in the contact list based on theinput voice.

In operation 740, the device 10 may send the name of the recipient andthe contact information to the bank server 20 along with the amount ofmoney included in the voice input. The contact information may besearched for by the name of the recipient or entered via voice input bythe user.

In operation 750, the device 10 may receive remittance details from thebank server 20. The bank server 20 may search for an account number ofthe recipient by using the name and the contact information of therecipient and transmit to the device 10 the remittance detailsincluding, without limitation, the name of the recipient, the accountnumber, and the amount of money.

In operation 760, the device 10 may approve (authenticate) theremittance details. The device 10 may approve the remittance details byusing, without limitation, the user's voice input, fingerprint, irisscan, vein image, facial image, and/or a password. The user may confirmthe remittance details and use voice input to the device 10 to approvethe remittance details or allow the device 10 recognize the iris, thefingerprint, and the like. Also, when the user wears a wearable devicesuch as a smart watch, the user may perform authentication by using avein in the back of a user's hand by using the smart watch. For example,the user may manipulate the smart watch to identify the vein in the backof the hand and perform authentication so as to approve the remittancedetails.

FIG. 8 is a diagram illustrating a method for paying using voice inputaccording to another example embodiment. Referring to FIG. 8 , a usermay pay by using voice input.

The device 10 may display a screen on display 13 for paying for goods orservices purchased by the user on the Internet. For example, when theuser purchases a Galaxy Note 7, the device 10 may display a message‘Would you like to purchase a Galaxy Note 7?’

After checking payment details, the user may provide voice input to pay.For example, when the user inputs ‘pay with Samsung Card’, the device 10may recognize the user's voice as noted at item 1 in FIG. 8 . The usermay simply input ‘Pay’, and the device 10 may proceed with payment byusing a card previously used by the user to pay.

The device 10 may transmit card information of the user and paymentinformation to a card issuer server 30 as noted by item 2 in FIG. 8 .The card information of the user may include a card number, anexpiration date of the card, a password, and the like. The paymentinformation may include goods or services to be paid, sellerinformation, and the like.

The card issuer server 30 may confirm the card information and proceedwith payment as noted by item 3 in FIG. 8 . The card issuer server 30may transmit a payment completion message to the device 10 when paymentis completed. The device 10 may display the payment completion messageto notify the user that the payment has been completed normally.

As an example, if the user wears a smart watch and the user pays forgoods or services, the smart watch may automatically perform biometricauthentication on the user. For example, if the user is wearing thesmart watch, the smart watch may capture a vein of a user's wrist andperform vein authentication through a pattern of the captured vein.Therefore, the user may automatically pay through the smart watchwithout inputting the voice, a password, etc. separately. Morespecifically, the device 10 may determine whether the user wears thesmart watch when the user touches a payment button over the Internet. Ifthe user is wearing the smart watch, the device 10 may send a signal tothe smart watch to perform vein authentication. The smart watch maycapture the vein of the user under control of the device 10 and transmita result of performing vein authentication to the device 10. Further,the smart watch may transmit an image of the captured vein to the device10, and the device 10 may perform vein authentication. Veinauthentication may compare a registered vein image (or a vein pattern)with a captured vein image (or the vein pattern). When the user wearsthe wearable device, the device 10 may proceed with payment withoutreceiving a separate input from the user.

FIG. 9 is a diagram for illustrating the device 10 learning a paymentpattern according to an example embodiment. Referring to FIG. 9 , thedevice 10 may analyze payment patterns by analyzing various types ofsentences. Learning of the payment pattern may mean to identify andrecord a type of voice input that a user speaks when paying.

The user may say various types of sentences to pay. For example, theuser may say the following types of sentences.

-   1. Pay with Samsung Card-   2. Please pay with Samsung Card-   3. Pay with my card-   4. Proceed with payment

The device 10 may store in memory 12 an expression mainly (most often)said when the user pays and determine whether the user says the same orsimilar sentence as the stored sentence, and proceed with payment.

The device 10 may register card information of the user at the beginningof learning or request the card information from the user in order toobtain the card information that the user mainly uses. When the cardinformation of the user is registered, even if the user simply says “Paywith my card”, the device 10 may proceed with payment by using thepreviously registered card information of the user.

FIG. 10 is a flowchart illustrating a method for paying by voice inputaccording to an example embodiment. Referring to FIG. 10 , a user maypay for goods or services by using voice input.

In operation 1010, the device 10 may display payment details on display13.

In operation 1020, the device 10 may receive the user's voice input viamicrophone 14. The user may check the payment details, and may expresswhether or not to pay by providing a voice input. For example, the usermay say “Pay” when paying and say “Do not pay” when not paying.

In operation 1030, the device 10 may analyze the received voice input todetermine an intention of the user. The device 10 may analyze the voiceinput and determine whether the user would like to approve the displayedpayment details.

In operation 1040, the device 10 may perform user authentication byvoice input.

The device 10 may perform user authentication by determining whether thevoice input matches the user's voice (e.g., by comparing to a registeredvoice sample). The device 10 may determine whether the registered voicesample matches the input voice and, if so, proceed with payment. Thedevice 10 may perform user authentication through not only voice, butalso fingerprints, irises, veins, faces, or passwords.

In operation 1050, the device 10 may transmit payment information to acard company. If authentication is successful, the device 10 maytransmit the payment information and the card information to the cardcompany. The payment information may include goods, seller information,an amount of money, and the like. The card information may include acard number of the user, a password, an expiration date, and the like.

The device 10 may display a payment completion message upon completionof payment.

As described above, when the user purchases goods or services via theInternet, the user may purchase goods or services through voice input.

FIG. 11 is a block diagram of a processor 1300 according to some exampleembodiments.

Referring to FIG. 11 , the processor 1300 according to some exampleembodiments may include a data learner 1310 and a data recognizer 1320.

The data learner 1310 may learn a reference for determining a situation.The data learner 1310 may learn a reference for what data to use todetermine a predetermined situation and how to determine the situationby using data. The data learner 1310 may obtain data to be used forlearning, and apply the obtained data to a data recognition model thatwill be described below, thereby learning the reference for determiningthe situation.

The data learner 1310 may learn a data recognition model by using voiceinput or a sentence to generate the data recognition model set toestimate an intention of a user. At this time, the voice input or thesentence may include a voice uttered by the user of the device 10 or asentence with which a user's voice is recognized. Alternatively, thevoice or the sentence may include a voice uttered by the third party ora third party's voice.

The data learner 1310 may learn the data recognition model by using asupervised learning method using a voice or a sentence and a learningentity as learning data.

In an example embodiment, the data recognition model may be a model setto estimate an intention of the user to send money. In this case, thelearning entity may include, without limitation, at least one of userinformation, recipient information, remittance amount, and remittanceintention. The user information may include, without limitation,identification information (e.g., a name or a nickname) of the user oridentification information of an account of the user (e.g., an accountbank, an account name, an account nickname or an account number). Therecipient information may include, without limitation, identificationinformation (e.g., a name, a nickname or a phone number) oridentification information of an account of a recipient (e.g., anaccount bank, an account name, an account nickname, or an accountnumber). The remittance intention may include whether the user will sendmoney. For example, the remittance intention may include, withoutlimitation, remittance proceedings, remittance reservations, reservationcancellation, remittance holding, or remittance confirmation.

On the other hand, at least one learning entity value may have a valueof ‘NULL’. In this case, the value ‘NULL’ may indicate that there is noinformation about an entity value for the voice input or the sentenceused as learning data.

Specifically, if the voice input or the sentence for learning is ‘Send100 million won from A bank account to Samsung’, the learning entity is{user information: A bank, recipient information: Samsung, remittanceamount: 100 million won, remittance instruction: proceed withremittance}. As another example, if the voice input or the sentence forlearning is ‘Send 100 million won to Samsung’, the learning entity mayconsist of {user information: NULL, recipient information: Samsung,remittance amount: 100 million won, remittance instruction: proceed withremittance}. As another example, if the voice or the sentence forlearning is ‘Is it right to have sent 100 million won to Samsung?’, thelearning entity may consist of {user information: NULL, recipientinformation: Samsung, remittance amount: 100 million won, remittanceinstruction: confirm remittance}. As another example, if the voice orthe sentence for learning is ‘Cancel reservation to send 100 million wonto Samsung’, the learning entity may consist of {user information: NULL,recipient information: Samsung, remittance amount: 100 million won,remittance instruction: cancel reservation}.

In another example embodiment, the data recognition model may be a modelset to estimate a payment intention of the user. In this case, thelearning entity may include, without limitation, at least one of apayment card, a payment item, a payment method, and the paymentintention. The payment method may include, for example, payment in fullor the number of monthly installments. The payment intention may includewhether the user will pay. For example, the payment intention mayinclude payment proceeding, payment cancellation, payment holding, apayment method change, or payment confirmation.

Specifically, if the voice input or the sentence for the learning is‘pay in full with Samsung Card’, the learning entity may be composed of{payment means: Samsung card, payment item: NULL, payment method:payment in full, payment instruction: proceed with payment}. As anotherexample, if the voice input or the sentence for learning is ‘Pay in 10monthly installments’, the learning entity may be composed of {paymentmeans: NULL, payment item: NULL, settlement method: 10 monthlyinstallments, payment instruction: proceed with payment}. As anotherexample, if the voice input or the sentence for learning is ‘Cancel aprevious payment’, the learning entity may be composed of {paymentmeans: NULL, payment item: NULL, payment method: NULL, paymentinstruction: cancel payment}.

The data recognition model set to determine the remittance intention ofthe user and the data recognition model set to determine the paymentintention of the user may be the same recognition model or differentrecognition models. Alternatively, each of the data recognition modelsmay include a plurality of data recognition models. For example, theintention of the user may be determined by using the plurality of datarecognition models customized for each environment considering a useenvironment (for example, a use time or a place of use) of the user.

The data recognizer 1320 may determine a situation based on data. Thedata recognizer 1320 may recognize the situation from predetermined databy using the learned data recognition model. The data recognizer 1320may determine a predetermined situation based on predetermined data byobtaining the predetermined data according to a predetermined referenceby learning and using the data recognition model by using the obtaineddata as an input value. Further, a resultant value output by the datarecognition model by using the obtained data as the input value may beused to update the data recognition model.

The data recognizer 1320 may estimate an intention of the user byapplying the user's voice input or a sentence with which the user'svoice is recognized to the data recognition model. For example, the datarecognizer 1320 may apply the user's voice input or the sentence withwhich the user's voice is recognized to the data recognition model toobtain a recognition entity and provide the recognition entity to aprocessor of a device (e.g., the processor 11 of the device 10 of FIG. 2). The processor 11 may determine the intention of the user by using theobtained recognition entity.

In an example embodiment, the data recognition model may be a model setto estimate an intention of the user to send money. In this case, thedata recognizer 1320 may estimate the intention of the user to sendmoney by applying the user's voice input or the sentence with which theuser's voice is recognized to the data recognition model. For example,the data recognizer 1320 may obtain a recognition entity from the user'svoice input or the sentence with which the user's voice is recognized.The recognition entity may include, for example, at least one of userinformation, recipient information, a remittance amount, and aremittance instruction. The data recognizer 1320 may provide theobtained recognition entity to the processor 11. The processor 11 (or adialog management module of the processor 11) may determine theintention of the user based on the recognition entity.

If it is determined that the intention of the user includes no intentionto send money based on the recognition entity, the processor 11 may notperform a process for sending money. On the other hand, if it isdetermined based on the recognition entity that the intention of theuser is to send money, the processor 11 may perform the process forsending money.

At this time, if at least one of the values of the recognition entity is‘NULL’, the processor 11 may determine a value corresponding to a value‘NULL’ using history information of the user or preset information. Forexample, the processor 11 may determine a value corresponding to thevalue ‘NULL’ by referring to a recent remittance history. Alternatively,the processor 11 may determine a value corresponding to the value ‘NULL’by referring to information (for example, an account number, an accountbank, etc.) preset by the user in a preference setting.

If at least one of the values of the recognition entity is ‘NULL’, theprocessor 11 may request a value corresponding to the value ‘NULL’ fromthe user. For example, the processor 11 may control display 13 todisplay a sentence indicating that there is no information about atleast one of the user information, the recipient information, theremittance amount, or the remittance instruction. When the user inputsat least one piece of the above information by voice or other input(e.g., by virtual keyboard displayed on display 13), the processor 11may perform a process for sending money by using the recognition entityvalue obtained from the data recognizer 1320 and user input information.

In another example embodiment, the data recognition model may be a modelset to estimate a payment intention of the user. In this case, the datarecognizer 1320 may estimate the payment intention of the user byapplying the user's voice input or the sentence with which the user'svoice is recognized to the data recognition model. For example, the datarecognizer 1320 may obtain a recognition entity from the user's voiceinput or the sentence with which the user's voice is recognized. Therecognition entity may include, for example, at least one of paymentmeans, a payment item, a payment method and a payment instruction. Thedata recognizer 1320 may provide the obtained recognition entity to theprocessor 11. The processor 11 (or a dialog management module of theprocessor 11) may determine the intention of the user based on therecognition entity.

If it is determined based on the recognition entity that the intentionof the user is to not pay, the processor 11 may not perform a processfor payment. On the other hand, if it is determined based on therecognition entity that the intention of the user is to pay, theprocessor 11 may perform the process for payment.

On the other hand, if at least one of the values of the recognitionentity is ‘NULL’, the processor 11 may determine a value correspondingto the value ‘NULL’ using history information of the user or presetinformation. Alternatively, the processor 11 may request the user toinput a value corresponding to the value ‘NULL’.

At least one of the data learner 1310 and the data recognizer 1320 maybe manufactured as at least one hardware chip and mounted on anelectronic device. For example, at least one of the data learner 1310and the data recognizer 1320 may be manufactured as a dedicated hardwarechip for artificial intelligence (AI) or may be manufactured as a partof a conventional general purpose processor (e.g. a CPU or anapplication processor) or a graphics processor (e.g., a GPU) and may bemounted on various electronic devices as described above. In this case,the dedicated hardware chip for AI may be a dedicated processorspecialized for probability calculation, and have a higher parallelprocessing performance than the conventional general purpose processor,thereby quickly processing arithmetic operations in an AI field such asmachine learning.

In this case, the data learner 1310 and the data recognizer 1320 may bemounted on one electronic device or on separate electronic devices. Forexample, one of the data learner 1310 and the data recognizer 1320 maybe included in the electronic device, and the other may be included in aserver. The data learner 1310 and the data recognizer 1320 may providemodel information constructed by the data learner 1310 to the datarecognizer 1320 via wired or wireless communication. Data input to thedata recognizer 1320 may be provided to the data learner 1310 asadditional learning data.

Meanwhile, at least one of the data learner 1310 and the data recognizer1320 may be implemented as a software module. When at least one of thedata learner 1310 and the data recognizer 1320 is implemented as asoftware module (or a program module including an instruction), thesoftware module may be stored in non-transitory computer-readable media.Further, in this case, the at least one software module may be providedby an operating system (OS) or by a predetermined application.Alternatively, some of the at least one software module may be providedby the OS, and the others may be provided by the predeterminedapplication.

FIG. 12 is a block diagram of the data learner 1310 according to someexample embodiments.

Referring to FIG. 12 , the data learner 1310 according to some exampleembodiments may include a data obtainer 1310-1, a preprocessor 1310-2, alearning data selector 1310-3, a model learner 1310-4, and a modelevaluator 1310-5. In some example embodiments, the data learner 1310 mayindispensably include the data obtainer 1310-1 and the model learner1310-4, and may selectively include at least one of or may not includeall of the preprocessor 1310-2, the learning data selector 1310-3, andthe model evaluator 1310-5.

The data obtainer 1310-1 may obtain data necessary for learning fordetermining a situation.

For example, the data obtainer 1310-1 may obtain voice data, image data,text data, biometric signal data, or the like. Specifically, the dataobtainer 1310-1 may obtain a voice input or a sentence for sending moneyor payment. Alternatively, the data obtainer 1310-1 may obtain voicedata or text data including the voice or the sentence for sending moneyor payment.

The data obtainer 1310-1 may receive data through an input device (e.g.,a microphone, a camera, a sensor, keyboard, or the like) of anelectronic device. Alternatively, the data obtainer 1310-1 may obtaindata via an external device (e.g., a server) that communicates with adevice.

The preprocessor 1310-2 may preprocess the obtained data so that thedata obtained for learning for determining the situation may be used.The preprocessor 1310-2 may process the obtained data in a predeterminedformat so that the model learner 1310-4, which will be described below,may use the obtained data for learning for determining the situation.For example, the preprocessor 1310-2 may extract learning entity valuesfrom the voice data according to the predetermined format. For example,when the predetermined format is composed of {user information,recipient information, remittance amount, and remittance instruction},or when the predetermined format is composed of {payment means, paymentitem, payment method, payment instruction}, the preprocessor 1310-2 mayextract the learning entity value from the voice data according to theformat. At this time, if the learning entity value is not extracted, thepreprocessor 1310-2 may cause display a specific entity value to be‘NULL’.

The learning data selector 1310-3 may select data required for learningfrom the preprocessed data. The selected data may be provided to themodel learner 1310-4. In this case, the data obtained by the dataobtainer 1310-1 or the data processed by the preprocessor 1310-2 may beprovided to the model learner 1310-4 as learning data. The learning dataselector 1310-3 may select data required for learning from thepreprocessed data according to a predetermined reference for determiningthe situation. The predetermined reference may be determined, forexample, considering at least one of attributes of the data, ageneration time of the data, a creator of the data, reliability of thedata, a target of the data, a generation region of the data, and size ofthe data. Alternatively, the learning data selector 1310-3 may selectthe data according to the predetermined reference by learning by usingthe model learner 1310-4, which will be described below.

The model learner 1310-4 may learn a reference on how to determine thesituation based on the learning data. Also, the model learner 1310-4 maylearn a reference on which learning data should be used for determiningthe situation. For example, the model learner 1310-4 may learn adetermination model according to a supervised learning method or anunsupervised learning method to generate a data recognition model forpredicting, determining, or estimating. The data recognition model maybe, for example, a model set for estimating a remittance intention of auser or a model set for estimating a payment intention of the user.

Further, the model learner 1310-4 may learn the data recognition modelused for determining the situation by using the learning data. The datarecognition model may be a pre-built model. For example, the datarecognition model may be a pre-built model by receiving basic learningdata (e.g., sample data, etc.).

The data recognition model may be constructed considering an applicationfield of the recognition model, a purpose of learning, or the computerperformance of the device. The data recognition model may be, forexample, a model based on a neural network. The data recognition modelmay be designed to simulate a human brain structure on a computer. Thedata recognition model may include a plurality of network nodes havingweights to simulate a neuron of a human neural network. The plurality ofnetwork nodes may establish a connection relationship to simulate asynaptic activity of a neuron sending and receiving signals via synapse.The data recognition model may include, for example, a neural networkmodel or a deep learning model developed from the neural network model.In the deep learning model, the plurality of network nodes may belocated at different depths (or layers) and may exchange data accordingto a convolution connection relationship. For example, a model such as aDeep Neural Network (DNN), a Recurrent Neural Network (RNN), or aBidirectional Recurrent Deep Neural Network (BRDNN) may be used as adata recognition model, but the present disclosure is not limitedthereto.

According to various example embodiments, when there are a plurality ofdata recognition models that are built in advance, the model learner1310-4 may determine a data recognition model having a high relationbetween input learning data and basic learning data as a datarecognition model to learn. In this case, the basic learning data may bepre-classified according to data types, and the data recognition modelmay be pre-built for each data type. For example, the basic learningdata may be pre-classified by various references such as a region wherelearning data is generated, a time at which the learning data isgenerated, a size of the learning data, a genre of the learning data, acreator of the learning data, a genre of an object in the learning data,etc.

Also, the model learner 1310-4 may learn the data recognition model byusing, for example, a learning algorithm including an errorback-propagation method or a gradient descent method.

Also, the model learner 1310-4 may learn the data recognition modelthrough supervised learning by, for example, using the learning data asan input value. Also, the model learner 1310-4 may learn the datarecognition model through unsupervised learning to find a reference fordetermining the situation by, for example, learning a type of datanecessary for determining the situation for itself. Also, the modellearner 1310-4 may learn the data recognition model throughreinforcement learning, for example by using feedback on whether aresult of determination of the situation based on the learning iscorrect.

The learning data may include a user's voice input or a third party'svoice input, a sentence via which the user's voice or the third party'svoice is recognized, a sentence entered by the user or the third party,and the like. Also, the learning data may include a learning entityassociated with the voice input or the sentence. Various examples oflearning entities are described in detail with reference to FIG. 11 ,and thus redundant descriptions thereof are omitted.

Further, when the data recognition model is learned, the model learner1310-4 may store the learned data recognition model. In this case, themodel learner 1310-4 may store the learned data recognition model in amemory of an electronic device (for example, memory 12 of theabove-described device 10) including the data recognizer 1320.Alternatively, the model learner 1310-4 may store the learned datarecognition model in a memory of an electronic device including the datarecognizer 1320 that will be described below. Alternatively, the modellearner 1310-4 may store the learned data recognition model in a memoryof a server connected to the electronic device (for example, theabove-described device 10) via a wired or wireless network.

In this case, the memory in which the learned data recognition model isstored may also store, for example, instructions or data associated withat least one other component of the electronic device. The memory mayalso store software and/or a program. The program may include, forexample, a kernel, middleware, an application programming interface(API), and/or an application program (or “application”).

The model evaluator 1310-5 may input evaluation data to the datarecognition model, and if a recognition result output from theevaluation data does not satisfy a predetermined reference, the modelevaluator 1310-5 may allow the model learner 1310-4 to learn again. Inthis case, the evaluation data may be predetermined data for evaluatingthe data recognition model.

For example, when the number or a ratio of the evaluation data whoserecognition result is not correct exceeds a preset threshold value inthe recognition result of the learned data recognition model for theevaluation data, the model evaluator 1310-5 may evaluate the learneddata recognition model as not satisfying the predetermined reference.For example, when the predetermined reference is defined as a ratio of2%, when the learned data recognition model outputs an incorrectrecognition result for evaluation data exceeding 20 pieces of evaluationdata among a total of 1000 pieces of the evaluation data, the modelevaluator 1310-5 may evaluate that the learned data recognition model isnot suitable.

On the other hand, when there are a plurality of learned datarecognition models, the model evaluator 1310-5 may evaluate whether eachof the learned data recognition models satisfies a predeterminedreference and may determine a model satisfying the predeterminedreference as a final data recognition model. In this case, when thereare a plurality of models satisfying the predetermined reference, themodel evaluator 1310-5 may determine any one model previously set indescending order of an evaluation score or a predetermined number ofmodels as the final data recognition model.

Meanwhile, at least one of the data obtainer 1310-1, the preprocessor1310-2, the learning data selector 1310-3, the model learner 1310-4, andthe model evaluator 1310-5 included in the data learner 1310 may bemanufactured in at least one hardware chip and mounted on an electronicdevice. For example, at least one of the data obtainer 1310-1, thepreprocessor 1310-2, the learning data selector 1310-3, the modellearner 1310-4, and the model evaluator 1310-5 may be manufactured as adedicated hardware chip for AI or may be manufactured as a part of aconventional general purpose processor (e.g. a CPU or an applicationprocessor) or a graphics processor (e.g., a GPU) and may be mounted onvarious electronic devices as described above.

Also, the data obtainer 1310-1, the preprocessor 1310-2, the learningdata selector 1310-3, the model learner 1310-4, and the model evaluator1310-5 may be mounted on one electronic device or may be mounted onseparate electronic devices. For example, some of the data obtainer1310-1, the preprocessor 1310-2, the learning data selector 1310-3, themodel learner 1310-4, and the model evaluator 1310-5 may be included inthe electronic device, and the others may be included in a server.

Also, at least one of the data obtainer 1310-1, the preprocessor 1310-2,the learning data selector 1310-3, the model learner 1310-4, and themodel evaluator 1310-5 may be implemented as a software module. When atleast one of the data obtainer 1310-1, the preprocessor 1310-2, thelearning data selector 1310-3, the model learner 1310-4, and the modelevaluator 1310-5 is implemented as a software module (or a programmodule including an instruction), the software module may be stored innon-transitory computer-readable media. Further, in this case, the atleast one software module may be provided by an operating system (OS) orby a predetermined application. Alternatively, some of the at least onesoftware module may be provided by the OS, and the others may beprovided by the predetermined application.

FIG. 13 is a block diagram of the data recognizer 1320 according to someexample embodiments.

Referring to FIG. 13 , the data recognizer 1320 according to someexample embodiments may include a data obtainer 1320-1, a preprocessor1320-2, a recognition data selector 1320-3, a recognition resultprovider 1320-4 and a model updater 1320-5. In some example embodiments,the data recognizer 1320 may indispensably include the data obtainer1320-1 and the recognition result provider 1320-4, and may selectivelyinclude at least one of the preprocessor 1320-2, the recognition dataselector 1320-2, and the model updater 1320-5.

The data obtainer 1320-1 may obtain data necessary for determining asituation. For example, the data obtainer 1320-1 may obtain a user'svoice input or a sentence with which the user's voice is recognized.Specifically, the data obtainer 1320-1 may obtain the user's voice inputor the sentence for performing remittance or payment. Alternatively, thedata obtainer 1320-1 may obtain voice data or text data including theuser's voice or the sentence for performing remittance or payment.

The preprocessor 1320-2 may preprocess the obtained data so that thedata obtained for determining the situation may be used. Thepreprocessor 1320-2 may process the obtained data into a predeterminedformat so that the recognition result provider 1320-4, which will bedescribed below, may use the obtained data for determining thesituation. For example, the preprocessor 1320-2 may extract learningentity values from the voice data according to the predetermined format.For example, the preprocessor 1320-2 may extract the learning entityvalues according to the format of {user information, recipientinformation, remittance amount, and remittance instruction} or {paymentmeans, payment item, payment method, payment instruction}.

The recognition data selector 1320-3 may select data necessary for thedetermination of the situation from the preprocessed data. The selecteddata may be provided to the recognition result provider 1320-4. Therecognition data selector 1320-3 may select a part or a whole of thepreprocessed data according to a preset reference for determining thesituation. The predetermined reference may be determined, for example,considering at least one of attributes of the data, a generation time ofthe data, a creator of the data, a reliability of the data, a target ofthe data, a generation region of the data and a size of the data.Alternatively, the recognition data selector 1320-3 may select dataaccording to the predetermined reference by learning by the modellearner 1310-4.

The recognition result provider 1320-4 may apply the selected data to adata recognition model to determine the situation. The recognitionresult provider 1320-4 may provide a recognition result according to adata recognition purpose. The recognition result provider 1320-4 mayapply the selected data to the data recognition model by using the dataselected by the recognition data selector 1320-3 as an input value.Also, the recognition result may be determined by the data recognitionmodel.

For example, when the data recognition model is a model set forestimating a remittance intention of the user, the recognition resultprovider 1320-4 may apply the user's voice input saying the sentence viawhich the user's voice input is recognized to the data recognition modelto estimate, infer or predict the remittance intention of the user.Alternatively, when the data recognition model is a model set forestimating a payment intention of the user, the recognition resultprovider 1320-4 may apply the user's voice input or the sentence withwhich the user's voice input is recognized to the data recognition modelto estimate (or infer or predict) the payment intention of the user.

The recognition result provider 1320-4 may obtain a recognition entityas a result of estimating the intention of the user. The recognitionresult provider 1320-4 may provide the obtained recognition entity to aprocessor (e.g., the processor 11 of the device 10 of FIG. 2 ). Theprocessor may determine the intention of the user based on therecognition entity and proceed with a process for sending money orpayment.

The model updater 1320-5 may update the data recognition model based onan evaluation of the recognition result provided by the recognitionresult provider 1320-4. For example, the model updater 1320-5 mayprovide the model learner 1310-4 with the recognition result provided bythe recognition result provider 1320-4 so that the model learner 1310-4may update the data recognition model.

Alternatively, the model updater 1320-5 may receive an evaluation (orfeedback) on the recognition result from a processor (for example, theprocessor 11 of the device 10 of FIG. 2 ). For example, the device 10may display remittance details according to the remittance intention ofthe user by applying a voice input by the user to the data recognitionmodel.

The user may approve the remittance details or refuse to approve theremittance details. For example, if the user approves the remittancedetails, the user may enter voice, fingerprints, an iris scan, a veinimage, a face image or a password. On the other hand, when the userrefuses to approve the remittance details, the user may select a cancelbutton, enter a voice requesting cancellation, or not perform any inputfor a predetermined period of time.

In this case, user feedback according to an approval or a rejection ofthe user may be provided to the model updater 1320-5 as an evaluation ofthe recognition result. That is, the user feedback may includeinformation indicating that a determination result of the datarecognizer 1320 is false or information indicating that thedetermination result is true. The model updater 1320-5 may update adetermination model by using the obtained user feedback.

Meanwhile, at least one of the data obtainer 1320-1, the preprocessor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model updater 1320-5 in the data recognizer1320 may be manufactured in at least one hardware chip and mounted on anelectronic device. For example, at least one of the data obtainer1320-1, the preprocessor 1320-2, the recognition data selector 1320-3,the recognition result provider 1320-4, and the model updater 1320-5 maybe manufactured as a dedicated hardware chip for AI or may bemanufactured as a part of a conventional general purpose processor (e.g.a CPU or an application processor) or a graphics processor (e.g., a GPU)and may be mounted on various electronic devices as described above.

Also, at least one of the data obtainer 1320-1, the preprocessor 1320-2,the recognition data selector 1320-3, the recognition result provider1320-4, and the model updater 1320-5 may be mounted on one electronicdevice or may be mounted on separate electronic devices. For example,some of the data obtainer 1320-1, the preprocessor 1320-2, therecognition data selector 1320-3, the recognition result provider1320-4, and the model updater 1320-5 may be included in the electronicdevice, and the others may be included in a server.

Also, at least one of the data obtainer 1320-1, the preprocessor 1320-2,the recognition data selector 1320-3, the recognition result provider1320-4, and the model updater 1320-5 may be implemented as a softwaremodule. When at least one of the data obtainer 1320-1, the preprocessor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model updater 1320-5 is implemented as thesoftware module (or a program module including an instruction), thesoftware module may be stored in non-transitory computer-readable media.Further, in this case, the at least one software module may be providedby an OS or by a predetermined application. Alternatively, some of theat least one software module may be provided by the OS, and the othersmay be provided by the predetermined application.

FIG. 14 is a diagram illustrating an example of learning and recognizingdata by interaction between a device 1000 and a server 2000 according tosome non-limiting embodiments.

The device 1000 may correspond to, for example, the device 10 of FIG. 2. The data obtainer 1320-1, the preprocessor 1320-2, the recognitiondata selector 1320-3, the recognition result provider 1320-4 and themodel updater 1320-5 of the data recognizer 1320 of the device 1000 mayrespectively correspond to the data obtainer 1320-1, the preprocessor1320-2, the recognition data selector 1320-3, the recognition resultprovider 1320-4, and the model updater 1320-5 of the data recognizer13320 of FIG. 13 . Also, the data obtainer 2310, the preprocessor 2320,the learning data selector 2330, the model learner 2340, and the modelevaluator 2350 of the data learner 2300 of the server 2000 respectivelycorrespond to the data obtainer 1310-1, the preprocessor 1310-2, thelearning data selector 1310-3, the model learner 1310-4, and the modelevaluator 1310-5.

The device 1000 may interact with the server 2000 through short-range orlong-distance communication. Connecting of the device 1000 and theserver 2000 to each other means that the device 1000 and the server 2000are directly connected to each other or connected to each other throughanother component (e.g., at least one of an access point (AP), a hub, arelay device, a base station, a router, and a gateway as a thirdcomponent).

Referring to FIG. 14 , the server 2000 may learn a reference fordetermining a situation, and the device 1000 may determine the situationbased on a learning result by the server 2000.

In this case, the model learner 2340 of the server 2000 may perform afunction of the data learner 1310 shown in FIG. 12 . The model learner2340 of the server 2000 may learn what data to use to determine apredetermined situation and how to determine the situation by usingdata. The model learner 2340 may obtain data to be used for learning,and apply the obtained data to a data recognition model to learn thereference for determining the situation. For example, the model learner2340 may learn the data recognition model by using a voice input or asentence to generate a data recognition model set to estimate anintention of a user. The generated data recognition model may be, forexample, a model set for estimating at least one of a remittanceintention of the user and a payment intention.

The recognition result provider 1320-4 of the device 1000 may determinethe situation by applying the data selected by the recognition dataselector 1320-3 to the data recognition model generated by the server2000. For example, the recognition result provider 1320-4 may transmitthe data selected by the recognition data selector 1320-3 to the server2000. The server 2000 may apply the data selected by the recognitiondata selector 1320-3 to the data recognition model to requestdetermination of the situation. Also, the recognition result provider1320-4 may receive from the server 2000 information about the situationdetermined by the server 2000. For example, when the selected dataincludes a user's voice input or a sentence with which the user's voiceis recognized, the server 2000 may apply the selected data to the datarecognition model set to estimate an intention of the user to obtain arecognition entity including the intention of the user. The server 2000may provide the obtained entity to the recognition result provider1320-4 as information on the determined situation.

As another example, the recognition result provider 1320-4 of the device1000 may receive the recognition model generated by the server 2000 fromthe server 2000 and determine the situation by using the receivedrecognition model. In this case, the recognition result provider 1320-4of the device 1000 may apply the data selected by the recognition dataselector 1320-3 to the data recognition model received from the server2000 to determine the situation. For example, when the selected dataincludes the user's voice input or the sentence with which the user'svoice is recognized, the recognition result provider 1320-4 of thedevice 1000 may apply the selected data to a data recognition model setto estimate an intention of the user received from the server 2000 toobtain a recognition entity including the intention of the user. Thedevice 1000 may then provide the obtained entity to a processor (e.g.,the processor 11 of FIG. 2 ) as information about the determinedsituation.

The processor 11 may determine a remittance intention of the user or apayment intention based on the recognition entity, and may perform aprocess for sending money or payment.

The device 10 according to an example embodiment may send money to arecipient by only a voice input.

The device 10 according to an example embodiment may send money to therecipient by transmitting a name of the recipient, a contact, and anamount of money to the bank server 20 without having to transmit anaccount number of the recipient.

The device 10 according to an example embodiment may pay by only a voiceinput.

FIGS. 15 and 16 are flowcharts of a network system using a datarecognition model according to some non-limiting example embodiments.

In FIGS. 15 and 16 , the network system may include first components1501 and 1601 and second components 1502 and 1602. Here, the firstcomponents 1501 and 1601 may be the device 1000, and the secondcomponents 1502 and 1602 may be the server 2000 that stores a dataanalysis model. Alternatively, the first components 1501 and 1601 may bea general purpose processor, and the second components 1502 and 1602 maybe an AI dedicated processor. Alternatively, the first components 1501and 1601 may be at least one application, and the second components 1502and 1602 may be an OS. That is, the second components 1502 and 1602 maybe components that are more integrated and dedicated and less delayedthan the first components 1501 and 1601 and have better performance andmore resources than the first components 1501 and 1601 and may processmany operations required for creating, updating, or applying a datarecognition model more quickly and effectively than the first components1501 and 1601.

In this case, an interface for transmitting/receiving data between thefirst components 1501 and 1601 and the second components 1502 and 1602may be defined.

For example, an application program interface (API) having learning datato be applied to the data recognition model as a factor value (or amedium value or a transfer value) may be defined. The API may be definedas a set of subroutines or functions that may be called for anyprocessing of any protocol (e.g., a protocol defined in the device 1000)to another protocol (e.g., a protocol defined in the server 2000). Thatis, an environment in which an operation of another protocol may beperformed in any one protocol through the API may be provided.

In FIG. 15 , the first component 1501 may analyze a remittance intentionof a user by using a data recognition model.

In operation 1511, the first component 1501 may receive a user's voiceuttered with the remittance intention.

In operation 1513, the first component 1501 may transmit the receivedvoice input or a sentence used to recognize the received voice to thesecond component 1502. For example, the first component 1501 may apply avoice input or a sentence as a factor value of an API function providedfor use of the data recognition model. In this case, the API functionmay transmit the voice input or the sentence to the second component1502 as recognition data to be applied to the data recognition model. Atthis time, the voice input or the sentence may be changed andtransmitted according to a promised communication format.

In operation 1515, the second component 1502 may apply the receivedvoice input or sentence to a data recognition model set to estimate theremittance intention of the user.

As a result of application, in operation 1517, the second component 1502may obtain a recognition entity. For example, the recognition entity mayinclude at least one of user information, recipient information (e.g., aname of a recipient), a remittance amount, and a remittance instruction.

In operation 1519, the second component 1502 may transmit therecognition entity to the first component 1501. At this time, therecognition entity may be changed and transmitted according to thepromised communication format.

In operation 1521, the first component 1501 may determine that theuser's voice input has the remittance intention based on the recognitionentity. For example, the first component 1501 may determine that theuser's voice has the remittance intention if the ‘proceed withremittance, the name of the recipient, and the remittance amount’ areincluded as remittance instruction values of the recognizing entity.

Here, operations 1513 to 1521 may correspond to an embodiment of aprocess in which the device 10 analyzes the received voice to determinethe remittance intention of the user in operation 720 of FIG. 2 .

If it is determined in operation 1521 that the user's voice has theremittance intention, the first component 1501 may search a contact listfor a contact corresponding to the name of the recipient included in therecognition entity in operation 1523.

In operations 1525, 1527 and 1529, the first component 1501 may approvedetails to send money to an account number of the recipient based on thefound contact of the recipient. The corresponding process corresponds tooperations 740 to 760 of FIG. 7 , and a redundant description thereofwill be omitted.

In FIG. 16 , the first component 1601 may analyze a payment intention ofthe user by using the data recognition model.

In operation 1611, the first component 1601 may provide payment details.For example, the first component 1601 may display the payment details ona screen or output the payment details by voice.

The user may check the payment details displayed on the screen, and mayexpress whether or not to pay by voice input.

In operation 1613, the first component 1601 may receive a user's voiceinput.

In operation 1615, the first component 1601 may transmit the receivedvoice input or a sentence that recognizes the received voice to thesecond component 1602. For example, the first component 1601 may apply avoice input or a sentence as a factor value of an API function providedfor use of the data recognition model. In this case, the API functionmay transmit the voice input or the sentence to the second component1602 as recognition data to be applied to the data recognition model. Atthis time, the voice input or the sentence may be changed andtransmitted according to a promised communication format.

In operation 1617, the second component 1602 may apply the receivedvoice or the sentence to a data recognition model set to estimate thepayment intention of the user.

As a result of application, in operation 1619, the second component 1602may obtain a recognition entity. For example, the recognition entity mayinclude, without limitation, at least one of payment means, a paymentitem, a payment method and a payment instruction.

In operation 1621, the second component 1602 may transmit therecognition entity to the first component 1601. At this time, therecognition entity may be changed and transmitted according to thepromised communication format.

In operation 1623, the first component 1601 may determine that theuser's voice has the payment intention based on the recognition entity.For example, if ‘cancel payment’ is included as a payment instructionvalue of the recognition entity, the first component 1601 may determinethat the user's voice has an intention not to proceed with payment. Onthe other hand, if ‘proceed with payment’ is included as the paymentinstruction value of the recognition entity, the first component 1601may determine that the user's voice has an intention to proceed withpayment.

Here, operations 1615 to 1623 may correspond to an embodiment of aprocess in which the device 10 analyzes the received voice anddetermines the payment intention of the user in operation 1030 of FIG.10 described above.

If it is determined that the user's voice input has the paymentintention, the first component 1601 may transmit payment information toa card company if user authentication through voice is successful inoperations 1625 and 1627. The corresponding process corresponds tooperations 1040 and 1050 of FIG. 10 , and a redundant descriptionthereof is omitted.

One or more example embodiments may be implemented using a recordingmedium including computer-executable instructions such as a programmodule executed by a computer system. A non-transitory computer-readablerecording medium may be an arbitrary available medium which may beaccessed by a computer system and includes all types of volatile andnon-volatile media and separated and non-separated media. Also, thenon-transitory computer-readable recording medium may include all typesof computer storage media and communication media. The computer storagemedia include all types of volatile and non-volatile and separated andnon-separated media implemented by an arbitrary method or technique forstoring information such as computer-readable instructions, a datastructure, a program module, or other data. The communication mediatypically include computer-readable instructions, a data structure, aprogram module, other pieces of data of a modulated signal, othertransmission mechanisms, and arbitrary information delivery media.

Furthermore, the method according to the embodiments may be provided asa computer program product.

The computer program product may include a software program, acomputer-readable storage medium storing the software program, or aproduct traded between a seller and a purchaser.

For example, the computer program product may include a product (e.g. adownloadable app) in the form of a software program distributedelectronically via the device 10 or a manufacturer of the device 10 oran electronic market (e.g. Google Play Store, App Store). For electronicdistribution, at least a part of the software program may be stored in astorage medium or may be created temporarily. In this case, the storagemedium may be a storage medium of a manufacturer or a server of theelectronic market, or a relay server.

Also, in the specification, “unit” may be a hardware component such as aprocessor or a circuit and/or a software component to be executed by ahardware component such as a processor.

The example embodiments described above are only illustrative, and itwill be understood by those of ordinary skill in the art that variouschanges in form and details may be made therein without changing thetechnical spirit of the present disclosure. Therefore, the exampleembodiments should be understood in the illustrative sense only and notfor the purpose of limitation. For example, each component described asa single type may be carried out by being distributed, and likewise,components described as a distributed type may also be carried out bybeing coupled.

It should be understood that example embodiments described herein shouldbe considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each exampleembodiment should typically be considered as available for other similarfeatures or aspects in other example embodiments.

While one or more example embodiments have been described with referenceto the figures, it will be understood by those of ordinary skill in theart that various changes in form and details may be made therein withoutdeparting from the spirit and scope as defined by the following claims.

What is claimed is:
 1. A device comprising: a memory; a microphone; andat least one processor configured to execute a program stored in thememory to control the device to perform operations for sending money toa recipient comprising: receiving, by the microphone, a voice input;analyzing the received voice input using a pre-built data recognitionmodel to estimate payment intention using an artificial intelligence(AI) algorithm, the data recognition model being customized for useenvironment of the device; obtaining, using the data recognition model,the payment intention of a user based on the analyzed voice inputcomprising at least one of a name of a recipient, remittance amount, ora remittance instruction; obtaining at least one contact informationfrom a stored contact list based on the name of the recipient identifiedfrom the received voice input; obtaining user input for selecting acontact information from among the at least one contact information;transmitting the name of the recipient and the contact informationselected based on the user input to a server together with theremittance amount; receiving, from the server, remittance informationgenerated based on the transmitted remittance amount; receiving afeedback input for either approving the received remittance informationto send the money to the recipient or rejecting the remittanceinformation to cancel sending of the money to the recipient; andupdating the data recognition model using the feedback input as anevaluation of payment intention of the user obtained using the datarecognition model, wherein the feedback input for approving theremittance information comprises user information including at least oneof a fingerprint, an iris scan, a facial image, a vein pattern imagine,or the voice of the user, and wherein the remittance information isidentified as being approved based on the user information having asimilarity equal to or greater than corresponding user informationstored in the memory.
 2. The device of claim 1, wherein the obtaining ofthe payment intention of the user comprises learning a pattern throughthe received voice input when the user sends money.
 3. The device ofclaim 1, wherein the at least one processor is configured to execute theprogram to control the device to perform further operations comprising:authenticating that the received voice input is a voice of a user of thedevice, wherein the payment intention of the user is obtained based onthe received voice input being authenticated as the voice of the user ofthe device.
 4. The device of claim 1, wherein the at least one processoris configured to execute the program to control the device to performfurther operations comprising: displaying the remittance information,and wherein the remittance information comprises an account number ofthe recipient.
 5. The device of claim 1, wherein the data recognitionmodel is a model based on an artificial intelligence (AI) algorithmusing learning entity values extracted from learning data comprisingvoice input or text, and wherein the learning entity values comprisevalues for at least one of user information, recipient information, aremittance amount, or a remittance instruction.
 6. The device of claim1, wherein the determining of the payment intention of the user is basedon recognition entity values obtained as a result of applying thereceived voice input to the data recognition model, wherein therecognition entity values comprise values for at least one of userinformation, recipient information, a remittance amount, or a remittanceinstruction.
 7. A paying method comprising: receiving, by a microphone,a voice input of a user; analyzing the received voice input using apre-built data recognition model to estimate payment intention using anartificial intelligence (AI) algorithm, the data recognition model beingcustomized for use environment of the device; obtaining, using the datarecognition model, the payment intention of a user based on the analyzedvoice input comprising at least one of name of a recipient, remittanceamount, or a remittance instruction; obtaining at least one contactinformation from a stored contact list based on the name of therecipient identified from the received voice input; transmitting thename of the recipient and contact information selected from the at leastone contact information based on user input to a server together withthe remittance amount; receiving, from the server, remittanceinformation generated based on the transmitted remittance amount;receiving a feedback input for either approving the received remittanceinformation to send money to the recipient or rejecting the remittanceinformation to cancel sending of the money to the recipient; andupdating the data recognition model using the feedback input as anevaluation of payment intention of the user obtained using the datarecognition model, wherein the feedback input for approving theremittance information comprises user information including at least oneof a fingerprint, an iris scan, a facial image, a vein pattern image, orthe voice of the user, and wherein the remittance information isidentified as being approved based on the user information having asimilarity equal to or greater than corresponding user information storein the memory.
 8. The paying method of claim 7, wherein the obtaining ofthe payment intention of the user comprises learning a pattern throughthe received voice input when the user sends money.
 9. The paying methodof claim 7, further comprising: authenticating that the received voiceis a voice of a user of a device, and obtaining the payment intention ofthe user based on the received voice input being authenticated as thevoice of the user of the device.
 10. The paying method of claim 7,further comprising: displaying the remittance information, wherein theremittance comprises information an account number of the recipient. 11.The paying method of claim 7, wherein the data recognition model is amodel based on an artificial intelligence (AI) algorithm using learningentity values extracted from learning data comprising voice input ortext, and wherein the learning entity values comprise values for atleast one of user information, recipient information, a remittanceamount, or a remittance instruction.
 12. The paying method of claim 7,wherein the determining of the payment intention of the user is based onrecognition entity values obtained as a result of applying the receivedvoice to the data recognition model, wherein the recognition entityvalues comprise values for at least one of user information, recipientinformation, a remittance amount, or a remittance instruction.
 13. Thepaying method of claim 7, further comprising: transmitting the receivedvoice input to an external server for requesting to estimate paymentintention of a user; and receiving the estimated payment intention fromthe external server, wherein the payment intention is estimated, by theexternal server, by analyzing the voice input using a pre-built datarecognition model to estimate the payment intention using an artificialintelligence (AI) algorithm.
 14. A computer program product comprising anon-transitory computer-readable storage medium storing instructionsconfigured to, when executed, cause a device to perform: receiving, by amicrophone, a voice input; analyzing the received voice input using apre-built data recognition model to estimate payment intention using anartificial intelligence (AI) algorithm, the data recognition model beingcustomized for use environment of the device; obtaining, using the datarecognition model, the payment intention of a user based on the analyzedvoice input comprising at least one of a name of a recipient, remittanceamount, or a remittance instruction; obtaining at least one contactinformation from a stored contact list based on the name of therecipient identified from the received voice input; obtaining user inputfor selecting a contact information from among the at least one contactinformation; transmitting the name of the recipient and the contactinformation selected based on the user input to a server together withthe remittance amount; receiving, from the server, remittanceinformation generated based on the transmitted remittance amount;receiving a feedback input for either approving the received remittanceinformation to send money to the recipient or rejecting the remittanceinformation to cancel sending of the money to the recipient; andupdating the data recognition model using the feedback input as anevaluation of payment intention of the user obtained using the datarecognition model, wherein the feedback input for approving theremittance information comprises user information including at least oneof a fingerprint, an iris scan, a facial image, a vein pattern image, orthe voice of the user, and wherein the remittance information isidentified as being approved based on the user information having asimilarity equal to or greater than corresponding user information storein the memory.